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FOREWORD 


This is the updated Evaluation Criteria Plan for the Space Transportation 
Main Engine Configuration Study and has been prepared as part of Task 3.0 of 
Contract NAS8-36867 (A Prime). The work is being performed by the Aerojet 
TechSystems Company for the NASA - Marshall Space Flight Center. 

The program objective is to identify candidate main engine configurations 
which enhance launch vehicle performance, operation and cost. These candidate 
configurations will be evaluated and the configuration(s) which provide signi- 
ficant advantages over existing systems will be selected for consideration for 
the next generation launch vehicles. 

The NASA-MSFC Project Manager is Mr. J. Thompson. The ATC Program 
Manager is Mr. T.C. Lacefield and the ATC Study Manager is Mr. E.K. Bair. 

The Evaluation Criteria Plan is Volume 2 of the Configuration Evaluation 
and Criteria Plan, Contract Data Requirement DR-9. Volume 1 is the System 
Trades Study and Design Methodology Plan, it has not required revision for the 
A Prime portion of the STME Configuration Study. 
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I. INTRODUCTION 


The unbiased selection of the Space Transportation Main Engine (STME) 
configuration requires that the candidate engines be evaluated against a 
predetermined set of criteria which must be properly weighted to emphasize 
critical requirements defined prior to the actual evaluation. Since the 
optimum configuration is a compromise between engine and airframe design, the 
criteria and relative weighting of the criteria involves a team effort between 
ATC, MSFC and the Space Transportation Architecture Study (STAS) contractors. 
The evaluation and selection process, Figure 1, involves the following func- 
tions: (1) determining if a configuration can satisfy basic STME requirements 
(yes/no) (2) defining the evaluation criteria, (3) selecting the criteria's 
relative importance or weighting, (4) determining the weighting sensitivities 
and (5) establishing a baseline for engine evaluation. The criteria weighting 
and sensitivities are cost related and are based on mission model and vehicle 
requirements. 

During Phase A of the STME study a Gas Generator Cycle engine was 
selected for conceptual design, with emphasis on reusability, reliability and 
low cost while achieving good performance. In Phase A Prime of the study 
emphasis will focus on expendable application of the STME while maintaining 
low cost and high reliability. 

This update of the Configuration Evaluation and Criteria Plan reflects 
the desire for an expendable engine and will also consider the effect of 
variable production rates. 

The STME/GG defined in Phase A will be used as a starting point for the 
A Prime study. The various configurations of this engine identified during 
the A Prime study, will be evaluated using the updated evaluation plan 
described in this document. 

The basic vehicle is a two stage LOX/HC (STBE) , L0X/LH 2 (STME) parallel 
burn vehicle capable of placing 150,000 lbs in low earth orbit (LEO).n The 
mission model calls for placement of payloads in LEO starting in the 1995 to 
1998 time frame. Each vehicle will utilize four STME's. 
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Figure 1. Configuration Evaluation and Selection Plan 
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I, Introduction (cont.) 


The STME has a normal power level (NPL) thrust of 435K lbf (vacuum) and 
an emergency power level (EPL) thrust 580K lbf (vacuum). The mission burn 
time is 520 seconds with a sea level ignition. 
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II. EVALUATION CRITERIA 


The evaluation criteria define the significant functions that are 
required to properly evaluate an engine system. These criteria include all 
the significant items covered by the STAS studies in the architecture evalua- 
tion as well as items considered significant by ATC. The criteria must allow 
evaluation from both an engine and vehicle system point of view for proper 
integration into a complete system. 

A. YES/NO SCREEN EVALUATION 

Initially, the engine concept must pass an evaluation relative to 
"yes/no" type criteria. A concept had to judged as a "yes" in all areas in 
order to be given further consideration. These criteria are: 

Safety 

Maximum Envelope 

Gimballing Capability 

Sea Level and Altitude Start Capability 

Single or Multi Engine Application 

Expendable or Reusable (Expendable for A Prime) 

Throttling Capability 

One Hundred Mission Capability (Expendable for A Prime) 

IOC Compatibility 

Stage Combustion and Gas Generator Cycle STME's passed this part of the evalu- 
ation and were then assessed on a quantitative basis during Phase A. 

B. QUANTITATIVE (COST BASED) SCREEN EVALUATION 

The quantitative criteria screen evaluation for the A Prime configura- 
tions, is based on cost and is divided into the following five categories: 
categories: (1) Performance and weight, (2) Development, (3) Production, (4) 
Facilities, and (5) Operation and Support, see Table I. The original evalua- 
tion criteria included "availability" but since all configuration to be con- 
sidered in the A Prime Phase will have the same availability this criteria was 
deleted. 
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TABLE I 


Category 

Performance 

Development 

Production 

Facilities 

Operation & 


QUANTITATIVE EVALUATION CRITERIA 

Criteria Subcriteria 


and Weight 


Isp 

Weight 

Development Time and 
Risk, Reliability 
and Safety 


Support 


Development 

Launch/GSE 

Installation & 
Checkout Launch 
Support 


Component 

Engine 
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II, B, Quantitative (Cost Based) Screen Evaluation (cont.) 


Performance and Weight 

The performance and weight category includes specific impulse and engine 
weight. The specific impulse represents a major factor in engine cost and 
complexity and vehicle system design. The engine propellant, efficiency, 
cycle, mixture ratio and chamber pressure are the primary factors in determin- 
ing specific impulse. 

Engine weight is dependent on thrust and chamber pressure require- 
ments. The weight is not a totally dominant factor in vehicle design and the 
airframe contractors are willing to sacrifice some engine weight to enhance 
operations, reliability and life while reducing costs. 

Development (DDT&E) 

The development category includes the factors that determine the DDT&E 
costs for developing an engine system and include manpower, hardware and 
testing. Engine cycle, thrust level, chamber pressure, propellant selection 
and life are important factors in determining development costs. Addition- 
ally, technology availability and development risk must be considered in this 
category. Reliability and safety features are incorporated during the design 
phase and verified during development. 

Production 


The production category includes the factors that determine the produc- 
tion cost of an engine system. Component weight and complexity relationships 
are used to determine their unit cost. These costs are summed and an assembly 
cost added to yield the overall engine cost. Variable production rates are 
considered through the use of learning curve relationships which consider the 
type of hardware being produced as well as the quantities. 
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II, B, Quantitative (Cost Based) Screen Evaluation (cont.) 


Facilities 


The facilities category determines the development and launch/ground 
support requirements for the engine development, acceptance and use. The 
development facilities are dependent on engine cycle propellants and chamber 
pressure. The launch and ground support criteria is dependent on propellant 
selections and engine cycle. 

Operation and Support 

The operation and support category includes the criteria involved with 
defining the operations cost of an engine. For an expendable application this 
includes installation, checkout and launch support. 
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III. "YES/NO" EVALUATION CRITERIA 


As discussed in Section II, the initial engine evaluation was a "yes/no" 
screening which required that a concept pass every element in the criteria to 
be considered further. All configuration considered in Phase A Prime have 
passed these criteria since they will be derived from the gas generator cycle 
engine selected in Phase A. 
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IV. QUANTITATIVE CRITERIA WEIGHTING 


The mission model assumed for use in developing the quantitative evalua- 
tion criteria will account for a variable fleet size. The first mission 
occurs between 1995 and 1998. For the upper stage, four engines are assumed 
with a burn time totaling 520 seconds for each mission. 

For use in the evaluations, it was assumed that combined capability of 
existing NSTL and other (new or existing) facilities would be in place and 
operation in time to support the STME development, qualification and produc- 
tion acceptance test schedule demands. 

The quantitative criteria weighting represents the relative importance 
of the defined categories and criteria used to evaluate an engine system. The 
STAS contractors recommend costing the criteria categories to establish their 
relative importance, and this is the procedure which will be used. The cost 
values for each category are shown in Table II and were estimated, using cost 
relationships developed in Phase A. The relative weighting of each category 
is a function of the number of missions anticipated and Table II reflects this 
relationship for some selected mission quantities. During the actual evalua- 
tion the baseline weighting values will be determined using the percentage 
relationships shown in Figure 2, depending on the number of missions selected. 

The performance and weight criteria are based on the effect of Isp and 
engine weight. The performance and weight cost is determined based on the 
potential loss of revenue due to a lower performing engine (-10 sec max) and 
the impact of added engine weight (1600 lbs, max, total). Because of added 
propellant or engine weight, a corresponding payload loss is incurred with an 
attendant loss in revenue. This is applied across the entire mission model. 

The DDT&E baseline cost is $1.5 x 10^ based on Phase A results and the 
unit cost (first) is set at $17 x 10®. A learning factor is applied to the 
quantities required to support the missions. Figure 3 shows the learning 
curve relationship; a. 9 learning curve was used. 
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♦CATEGORY COST AND WEIGHTING 
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Figure 2. Category Cost Relationships vs. Number of Missions 
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Figure 3. Learning Curve 







IV, Quantitative Criteria Weighting (cont.) 

The operations and support evaluation criteria are based upon an esti- 
mated installation and checkout time of 500 hours per engine. The facility 
costs were estimated assuming the construction of five engine test stands, an 
engine component test facility for pump, chamber and GC/preburner development 
and GSE requirements. 

After developing a cost for each category as discussed above, a percen- 
tage of the total cost was assigned to each category. The individual category 
costs and weighting are shown on Table II. The evaluation format used (for a 
fixed number of missions) is shown in Figure 4. 

Once the criteria weight is determined, for a given number of missions, 
it is then used as a basis for assessing the impact of the various engine 
concept features. The actual score achieved by a concept is arrived at using 
the baseline value for that category and adjusting it based on the actual 
value of that particular concept feature. 

As an example the value of performance, Isp, is derived from the esti- 
mated dollar value of a 10 second swing In performance; from 450 to 440 
seconds. At 450 seconds the candidate would receive the full value of the 
weighting value; as the performance diminishes towards 440 seconds the value 
would approach zero; ie actual value is equal to the weighting value times the 
quantity of the actual Isp minus 440 divided by 10. 

Actual Value = Weighting Value ( A £ tua1 ^ s P ~ AA0 ) 

The scoring system Is set-up to be open ended and allows for scores 
which may exceed the weighting value or are negative in value. 


13 


c 

o 

*35 

(0 


x 

X 

o 

LL 


/A 

1 

1 

I 

i 

1 

1 

1 

W 

+-» 

a 

a> 

o 

c 

o 

o 

1 

1 

i 

i 

1 

1 


1 

1 


i 


1 

l 

o 

c 

D) 

c 

UJ 

1 

1 

l 

i 

1 

1 

■ 








> 


UJ 

00 

h 






5 

a 

o 

6 

CO 



00 

a 

cc 

< 

0. 

o 

is 

a. 

■ 

> 

a 

• 

> 

a. 

■ 

> 

LL 

■ 

> 

o 

> 

o 

i— 

II 


£ 

=5 


BS 




</) 






(0 







VMM 

O) 






a> 

*a> 






+-» 

£ 






o 

"D 








C 







□ 

TO 






■ 1 

0 

0> 

*-» 

c 






a 

a> 

c 


<0 


CO 

3 

CO 

c 

TO 

E 

E 

CL 

O 

o 

■M 

o 

(/) 

a> 

c 

o 

•M 

_j 

> 

UJ 

In 

o 

»*- 

0) 

> 

3 

"O 

o 

a 

TO 

a) 

< 

h 



a> 

a> 

In 

TO 

a. 

o 



a. 

a 

0. 

Li. 

O 

i- 


(0 

E 

w 

O 

u. 

e 

o 

TO 

3 

TO 

> 

111 

** 

a 

o 

o 

c 

o 

O 

a> 

c 

5 ) 

c 

UJ 

a> 


14 











IV, Quantitative Criteria Weighting (cont.) 


Performance 


The weighting for the performance criteria is based on the impact the 
variation in performance will have on payload delivery capability. (In this 
case, a total performance variation of 10 seconds of Isp was assumed). Using 
a modified aV requirement, which accounts for gravity, drag and thrust losses, 
and an assumed lift-off thrust to weight ratio of 1.3, the propellant differ- 
ence imposed by the 10 second performance variations was determined. This was 
converted to equivalent payload assuming a $500/1 b to LEO delivery cost. 

The cost effect of engine weight was based on a potential total engine 
weight variance of 1600 lbs, for all four engines. This was then equated to 
loss of payload. 

Within this category the value of performance accounts for 84% of the 
total weighting value while the weight Impact is 16%. The performance evalu- 
ation criteria is graphically shown in Figure 5. 

Development 

The weighting for the development criteria is based on the DDT&E cost 
determined in the Phase A effort which was $1.5B. This figure is approxi- 
mately evenly divided between engineering support, development hardware and 
development testing. 

The testing assumes that their are 1000 tests costing $500,000 each. 

The attendant reliability associated with the 1000 tests is .99. Figure 6 
shows the relationship between reliability and the number of development tests 
required. If the desired reliability can be achieved in fewer tests then the 
DDT&E may be reduced; yielding an Improved score in this category. The scor- 
ing system also allows for variations in reliability requirements. 
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.84 x Weighting Value (P&W) 


Evaluation Criteria 


Weighting 
Value 

Performance | S p 84% ofW.V. (P&W) 

and 

Weight Weight 16% of W.V. (P&W) 



Figure 5. Performance and Weight Criteria Evaluation 
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Reliability is Achieved through Development Testing 

Figure 6. Reliability as a Function of Development Test Quantity 



IV, Quantitative Criteria Weighting (cont.) 


The hardware costs are based on development engines costing $20M each 
(includes development features, special instrumentation, etc.)* If the hard- 
ware cost changes then the score is adjusted to reflect the effect. 

The scoring relationship is similar to that use in the Phase A evalua- 
tion in that a DDT&E cost of S1.5B will receive a score of zero. A lower 
DDT&E cost will result in a positive score and a higher cost in a negative 
value. 


. . , .. , ...... ... ,$1.5B - Actual DDT&E $» 

Actual Value = Weighting Value (- — -) 

This relationship is shown graphically in Figure 7. 

Product i on 


A first (production) baseline cost of $17M per engine assembly will be 
used in this part of the evaluation. This value was developed in Phase A 
using the component cost, weight and complexity relationships shown in 
Figure 8. Here predicted component weights (from power balance program or by 
actual weights calculations) are used in conjunction with relative complexity 
factors to determine a cost per pound and subsequently the actual component 
cost. 


The component costs are summed and an assembly cost added to arrive at 
the total engine cost. 

As the quantities change the unit cost is adjusted by a learning curve 
as shown in Figure 3. For aerospace hardware a learning curve factor of .9 is 
assumed. 

The unit cost is multiplied by the quantity required to arrive at a 
production cost. 


18 


Weighting Value 



Figure 7. Development Evaluation 
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IV, Quantitative Criteria Weighting (cont.) 


The value of the criteria weighting factor is equal to the weighting 
factor (for the quantities envisioned) times the basic unit cost (S17M x 
L.C.F.) minus the estimated cost divided by the basic unit cost. 


Actual Value = Weighting Value 
Figure 9 shows this relationship. 


/ Basic Unit Cost - Actual Unit Cost x 
' Basic Unit Cost 


Operation 

For the expendable STME the baseline operations and support are equated 
to a cost of 500 hours per engine at a rate of $70/hr. As the time required 
or hourly rate change the value changes and the weighting value (for the 
quantity of engine being examined) can be determined. Actual Value = 
Weighting Value ( Base C ° --- t ) . The relationship is as seen in 

Figure 10. 

Facilities 


The weighting for the facility category is based on the cost of the new 
facilities involved with the development and launch support of the STME. 

These costs are estimated to be approximately $200 million which would yield 
weighting factors which are related to engine quantities, as shown in Table II 
and Figure 1. 

The estimated facilities cost for a particular design are assigned 
weighting value scores in a manner similar to those previously discussed. 
Actual Value = Weighting Value ( $200 $ M 2 ~ 0 ^ c - ^ ua1 - - Co - s - t -) . Figure 11 shows this 
relationship. 
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Figure 10. Operations Criteria Evaluation 
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Figure 11. Facilities Criteria Evaluation 
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V. BASELINE ENGINE 


The engine scores are determined by the addition of the weighting values 
for each criteria, which totals 100 points for the baseline. The engine 
design which provides the highest performance for the lowest price with the 
lowest operation and support costs represents the optimum choice on a life 
cycle cost basis. Rating the engines against the maximum value for a given 
category results in obtaining a clear perspective on the relative strengths 
and weaknesses of a given candidate configuration. 

Judging an engine based on its score for a given criteria, while varying 
the number of engines, is not valid because the category weighting values 
change as the number of engines change . However an assessment based on the 
total for all criteria categories is valid since the assessment is based on a 
total maximum score of 100 for all cases, independent of the number of engines 
involved. 
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